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Abstract 

We introduce a distributed source coding scheme called successive Wyner-Ziv coding. We show that any point 
in the rate region of the quadratic Gaussian CEO problem can be achieved via the successive Wyner-Ziv coding. The 
concept of successive refinement in the single source coding is generalized to the distributed source coding scenario, 
which we refer to as distributed successive refinement. For the quadratic Gaussian CEO problem, we establish a 
necessary and sufficient condition for distributed successive refinement, where the successive Wyner-Ziv coding 
scheme plays an important role. 



Index Terms 



> 

C^ ■ CEO problem, contra-polymatroid, rate splitting, source splitting, successive refinement, Wyner-Ziv coding. 

o 

\Q . I. Introduction 

O 

"^ I The problem of distributed source coding has assumed renewed interest in recent years. Many practical compres- 

sion schemes have been proposed for Slepian-Wolf coding (e.g. [1], [2] and the reference therein) and Wyner-Ziv 
coding (e.g. [3] and the reference therein), whose performances are close to the fundamental theoretical bounds [4], 
^ ■ [5]. Therefore it is of interest to reduce the general distributed source coding problem to these well-studied cases. 



Given L i.i.d. discrete sources Xi,X2, • • • ,Xl, the Slepian-Wolf rate region is the union of all the rate vectors 
(i?i, i?2, • • • ,Rl) satisfying 

^ i?, > H{Xa\Xx^\_a), V nonempty set A C J^, (1) 

ieA 

where X^ = {1, 2, • • • ,L} and Xj\ = {Xi}i^j\. The Slepian-Wolf reigon is a contra-polymatroid [6], [7] with LI 
vertices. Specifically, if vr is a permutation on T^, define the vector (_Ri(7r), R2{tt), • • • , Rl{tt)) by 

R-K(i){T^) = -ff(-'^7r(i)|-'^7r(i+l), • • ■ 7-''^7r(L)), i=l,---,i-l, (2) 

i?,(i)(7r) = i/(X,(L)). (3) 
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Then (i?i(7r), i?2(7r), • • • , Rl{tt)) is a vertex of the Slepian-Wolf region for every permutation tt. It is known that 
vertices of the Slepian-Wolf region can be achieved with a complexity which is significantly lower than that of a 
general point. It was observed in [8] that by splitting a source into two virtual sources one can reduce the problem 
of coding an arbitrary point in a i-dimensional Slepian-Wolf region to that of coding a vertex of a (2L — 1)- 
dimensional Slepian-Wolf region. The source-splitting approach was also adopted in distributed lossy source coding 
[9]. In the distributed lossy source coding scenario, we shall refer to source splitting as quantization splitting (from 
the encoder viewpoint) or description refinement (from the decoder viewpoint) since it is the quantization output, 
not the source, that gets split. Finally we want to point out that the source-splitting idea has a dual in the problem 
of coding for multiple access channels, which is referred to as rate-splitting [10]-[13]. 

The rest of this paper is divided into 3 sections. In Section II, we introduce a low complexity successive Wyner- 
Ziv coding schemem and prove that any point in the rate region of the quadratic Gaussian CEO problem can be 
achieved via this scheme. The duality between the superposition coding in multiaccess communication and the 
successive Wyner-Ziv coding is briefly discussed. The concept of distributed successive refinement is introduced in 
Section III. The quadratic Gaussian CEO problem is used as an example, for which the necessary and sufficient 
condition for the distributed successive refinement is established. We conclude the paper in Section IV. 

In this paper, we use boldfaced letters to indicate (n-dimensional) vectors, capital letters for random objects, 
and small letters for their realizations. For example, we let X = (^(1), • • • , X{n))'^ and x = (x{l), ■ ■ ■ , x{n)Y' . 
Calligraphic letters are used to indicate a set (say. A). We use Ua to denote the vector (t/i)ie^ with index i in an 
increasing order and use Ua,b to denote {UA,j)jeB '• For example, if A = B = {1, 2}, then C/4 = {Ui, U2) and 
Ua.b ~ {Ui,i: t/2,1, C/1.2, U2.2)- Here Ui (and Uij) can be a random variable, a constant or a function. We let f7_4 
be a constant if A is an empty set. We use 2k to denote the set {1, 2, • • • , K} for any positive integer K. 

II. Successive Wyner-Ziv Coding Scheme 

In this paper, we adopt the model of the CEO problem. But some of our results also hold for many other 
distributed source coding models. The CEO problem has been studied for many years [14]-[16]. Here is a brief 
description of this problem (also see Fig. 1). 

Let {X(t), Yi(t), • • • ,yL(i)}t^i be a temporally memoryless source with instantaneous joint probability dis- 
tribution P(x,yi, ■ ■ ■ jUl) on A" X 3^1 X ••• X y^, where X is the common alphabet of the random variables 
X{t) for t = 1, 2, • • • , and yi (z = 1, 2, • • • , L) is the common alphabet of the random variables Yi{t) for t = 
1,2,---. {X{t)}^i is the target data sequence that the decoder is interested in. This data sequence cannot be 
observed directly. L encoders are deployed, where encoder i observes {Yi{t)}^^, i = 1,2,- ■ ■ , L. The data rate at 
which encoder i {i = 1,2,- ■ ■ ,L) may communicate information about its observations to the decoder is limited to 
Ri bits per second. The encoders are not permitted to communicate with each other. Finally, the decision {X{t)}^i 
is computed from the combined data at the decoder so that a desired fidelity can be satisfied. 

' Here the elements of A- and B are assumed to be nonnegative integers. 
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Fig. 1 . Model of the CEO problem 

Definition 2.1: An i-tuple of rates Rij^ is said to be I?-admissible if for all e > 0, there exists an no such that 
for all n > no there exist encoders: 

/i"':3^r^{l,2,---,L2"('''+^^j}, * = 1,2, •..,£, 

and a decoder: 

gin) : ji^ 2, • • • , [2"(-Ri+^)j I X |l, 2, • • • , [2"(^^+^)j | ■ • • x {l, 2, • • • , [2"(-«^+^)j | 

such that 



A"" 



n 



J2d(xit),Xit) 



<D + e, 



where X — g^"^ ( /{"'(Yi), • • • , fj^' {Y m)) and d{-, ■) : X x X ^ [0, dmax] is a given distortion measure. We 
use Tl{D) to denote the set of all I?-admissible rate tuples. 
Definition 2.2 (Berger-Tung rate region): Let 



n{Wi^) - J i?i^ : ^ i?« > / (Fa; VK^ I T4^^c) ,V nonempty set ACiA 
I leA J 



(4) 



where Wi ^ Yi ^ (-^, ^t\{i}, W^IlXIj}) f^i"™ a Markov chain for all i eTl. The Berger-Tung rate region with 
respect to distortion D is 

TlBTiD) = conv I y n{WiJ , (5) 

where W(D) is the set of all Wj^ satisfying the following properties: 

(i) Wi ^ Yi ^ {X, Yx^\{i-}, Wx^\{i}) form a Markov chain for all i e Xl- 
(ii) There exists a function 

/ : Wi X • • • X Wl ^ -f 

such that Ed{X, X) < D, where X = .f{WxJ. 
It was shown in [17]-[19] that TZbt ^ Tl{D). The Berger-Tung rate region is the largest known achievable rate 
region for the general CEO problem although it was shown by Korner and Marton [20] that it is not always tight. 
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Computing the Berger-Tung rate region involves complicated optimization and convexification. Hence we shall only 
focus on TZ{Wx^). We will see that for the quadratic Gaussian CEO problem, the properties of the Berger-Tung 
rate region are determined completely by those of TZ{Wx^). 

It was proved in [21], [22] that TZ{Wxj^ ) is a contra-polymatroid with L! vertices. Specifically, if tt is a permutation 
on Tl, define the vector Rij^{n) by 

R^(i){T^) = I{y7:(e,\W^(e,\W^(^+i),--- ,W^(L)), * = I,--- ,-^- 1, (6) 

i?.(L)W = I{y.[L)\W^(L))- (7) 

Then Ri^^ (tt) is a vertex of TZ{Wx^ ) for every permutation n. The dominant face of TZ{Wx^ ) is the convex polytope 
consisting of all points Rx^ G Tl{Wx^) such that X]i=i ^i ~ ^O^Ilj ^II)- ^^Y rate tuple Rx,^ on the dominant 
face of TZ{Wxl ) has the property that 

R'x, < Ri, => R'x, = Rx, , yR'x, e nWx, ) 

where Rj^ < Rx,^ means R[ < Ri for all i G 2^. It is easy to check that the vertices of TZ{Wxl) are on its 
dominant face. For each vertex Rx,^ (tt), there exists a low-complexity successive Wyner-Ziv coding scheme which 
can be roughly described as follows: 
(i) Encoder Tr(L) employs conventional lossy source coding. Encoder 7r(i) {i = L — 1,L — 2,- ■ ■ ,1) employs 

Wyner-Ziv coding with side information W^(j^i), • • • , W^(^) at decoder. 
(ii) Decoder first decodes the codeword W^(^) from encoder tt{L), then successively decodes the codeword 

Wjrfi) (« = L — 1, L — 2, ■ • • ,1) from encoder 7r(i) with side information Wjrfi+i), • • • , W7r(L). 
Rate tuples on the dominant face other than these LI vertices were previously known to be attainable only by one 
of two methods. The first method known to achieve these difficult rate tuples was time sharing between vertices. 
This approach can require as many as L successive decoding schemes^, each scheme requiring L decoding steps. 
The second approach to achieve these rate tuples is joint decoding of all users. This is very difficult to implement 
in practice since random codes have a decoding complexity of the order of 2"^(^^i.'^^i\ where n is the block 
length. 

We will show that any rate tuple in TZ{Wxj^) can be achieved by a low-complexity successive Wyner-Ziv coding 
scheme with at most 2L — 1 steps. Without loss of generality, we only need to consider the rate tuple on the 
dominant face of TZ{Wx^)- Before proceeding to prove this result, we shall first give a formal description of the 
general successive Wyner-Ziv coding scheme. 

Let {Wi^Xm : W^2.i„ :••• jW^L.i™ ) jointly distributed with the generic source variables {X,Yx^) such that 
Wi^Xm- -^ Yi ^ {X,Yxj^\-^i-j,Wj,Xrn,,j G Ii\{i}) form a Markov chain for all i G Tl- Let cr be a permutation 
on {Wi^Xi, ■ ■ ■ , Wl,x^ } such that for all i G Xl, Wij is placed before Wi^k ii j < k (we refer to this type of 

^By Caratheodory's fundamental theorem [23], any point in the convex closure of a connected commpact set ^ in a d-dimensional Euclidean 
space can be represented as a convex combination of d + 1 or fewer points in the original set y4. 



permutation as the well-ordered permutation). Let {VFijI^ denote all the random variables that appear before Wij 
in the permutation cr. 

Random Binning at Encoder i: In what follows we shall adopt the notation and conventions of [24]. Let n-vectors 
Wi 1(1), • • • , Wi,i(Mi 1) be drawn independently according to a uniform distribution over the set T^{Wi,i) of e- 
typical Wi^i n-vectors, where M,,i = [2^(My.,w,,i)+4.i) ^^ That is, P{Wi^i{k) = w,,i) = 1/ || T,{W^a) ||, if 
Wi^i e T^{Wi^i), and = otherwise. Distribute these vectors into Ni,i bins: Bi^i{l), ■ ■ ■ ,Bi^i{Ni^i), such that 






< \B^s{b)\w,^ < 



M, 



1,1 



N.i 



1,2, 



• N,i, 



where Ni^i = L2("-f(^-^*'ilt^-'i>- )+^*'i)J and \B,^i{h)\w,i denotes the number of W,,i- vectors in B,,i(6). 

Successively from j = 2, j = 3, ■ • • , to j = mi, for each vector (fci, • • • , fcj-i) with fc^ G {1, 2, • • • , Mi^s} (s = 
1, • • • ,j — 1), let Wij(A:i, • • • , fcj-i, 1), • • • , Wij(A:i, • • • , fcj-i, -Mij) be drawn i.i.d. according to a uniform 
distribution over the set Tg{Wi,j\wi^i{ki), ■ ■ ■ ,Wij^i{ki, ■ ■ ■ , fcj-i)) of conditionally e-typical w^j's, conditioned 
on Wi^i(fci), • • • , Wij_i(/ci, • • • ,kj-i), and distribute them uniformly into Nij bins: Bij{l), ■ ■ ■ ,Bij{Nij) such 
that 



M, 



2J 



N, 



«j 



< \B^Ab)\w,., < 



M, 



«j 



N, 



«j 



6=l,2,...,iV,,,. 



Here M,,, = L2"(^(^-'^'-l^-^.-i)+^--)j, Ar,,, = [2"a(^..>^.J{M^...}.")+^.,.)j. Note: e,^^,e'^^j{i e Il,J e Im,) are 
positive numbers of the same order as e which can be made arbitrarily small as n ^ cxo. Furthermore, we require 
eij > e'ij for all i eIlJ e X„, 

Encoding at Encoder i: Given a y^ G y", find, if possible, a vector {k* i, • • ■ , k*^.) such that 

ThenfindbinsBj,i(6*i),Si^2(&j*2)r-- , ^j,™. (&,*„.) such that B^^ (6* j) contains Wjj(fc*i, •• • ,fc*^),j = 1,2,--- ,mi. 
Send (6*^, • • • , 6*„j.) to the decoder. If no such (fc*i, • • • , k*_„^,) exists, simply send (0, • • • , 0). 
We can see the resulting transmission rate of encoder i is 



^ / rrii \ mi mi ^ 



(8) 



Decoding: Given (6* j, • • • , 6*„.) for all z G X^, if (fe* ^, • • • , &*,„.) = (0, • • • ,0) for some i, declare a decoding 
failure. Otherwise decode as follows: 

Let (j{j) denote the f^ element in permutation a. Let si(j),S2(i) be the first and second subscript of cr(j), 
respectively. For example, if a{i) — VF3.2, then si{j) — 3, S2(.7) = 2. Decoder first finds '^si(i).s2(i){^si(i),s2(i)) 
in B{b* (.-^s ,^-.). Note: 52(1) = 1. Since -B(&* ^^ (2)) contains at most one vector, we have fcsi(i),s2(i) — 
K^{l),82{l)■ Successively from j = 2,j = 3,---, to j = Ef=i "^^ if i" ■S5iO),520)(^siO%s20))' ^^""^ ^^^^^^ ^ 
unique fcsi(j).s2 0) such that 
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decode '^si{j),s2{j){f^si{j),i7ks^(^j-)^27 ■ ■ ■ i ^si(j).s2(j))' otherwise declare a decoding failure. Note: e' is of the same 
order as e which can be made arbitrarily small as n — > cx). 

By the standard technique, it can be shown that Pr{kij = k* pVi e I^, j G 2™.) ^ 1 as n — > oo. Furthermore, 
by Markov Lemma [17], we have 



Pr ((X, W,^,(fc*i, A:*2, ' ' ' , Kj)'' ^ ^l,J £ X,n.) S T,, {X, W,,„i £ II,] S I„u)) -> 1 
as n — > oo. Hence for any function g : 11^=1 Y\T=i ^i.j ~^ '^' we have 



n 



J2 d (x(t),5 (W^^,,(fc- 1' K^^ • ■ • ' ^r,,' i)' » e II, J e x„,J) 



< Ed(X,.g(W,,,,i e Xi, j e X,„J) + e"dn 



with high probability, where Wij(fc*i, fc*2, • • • , K^ji^) is the i*'' entry of Wij(/c*;i, /c*2, • • • , A:*j) and e" is of 
the same order as e which can be made arbitrarily small as n ^ oo. 

It is easy to see that if we let Wl ■ = Wi,x- (Vi £ Zl, J G 2^. ), and replace Wij- by VF/ ■ in (|8}, i?i is unaffected. 
Hence there is no loss of generality to assume Wi^i -^ Wi_2 —>•■•—> Wi^rm ^ Yi,\/i G Xi,. We can view Wi j as 
a description of Yi, as j gets larger, the description gets finer. 

The above coding scheme can be interpreted in the following intuitive way: 

Encoderz first splits Ri into rrii pieces: rij = I{Yi; Wij\{Wij}'g:),yj G Xm^. Then successively from j = 1, j = 
2, • • ■ , to j = TOi, it uses a Wyner-Ziv code with rate r^.j to convey W^.j to decoder which has the side information 
{W^}~. Decoder recovers {Wij,j G Tniiji G 2^} successively according to the order in the permutation a. We 
can see that this scheme requires J2i=i "^j Wyner-Ziv coding steps. Thus we call it J2i=i "i^-successive Wyner-Ziv 
coding scheme. A similar successive coding strategy was developed in [25] for tree-structured sensor networks. 

The successive Wyner-Ziv encoding and decoding structure of the above scheme significantly reduces the coding 
complexity compared with joint decoding or time sharing scheme and makes the available practical Wyner-Ziv 
coding techniques directly applicable to the more general distributed source coding scenarios. Furthermore, the 
successive Wyner-Ziv coding scheme has certain robust property which is especially attractive in some applications. 
Since in the successive Wyner-Ziv coding scheme, encoder i essentially transmits its codeword in mi packets. Each 
packet contains a sub-codeword W^.j (j G ^m)- If a packet, say packet W^ ^ is lost in transmission, the decoder 
is still able to decode packets {Wik}^ ■ On the contrary, the jointly decoding scheme does not possess this robust 
property since any corruption in the transmitted codewords may cause a complete failure in decoding. 

We need introduce another definition before giving a formal statement of our first theorem. 

Definition 2.3: For any disjoint sets A,BCXi {A is nonempty), let 

n{WA\WB, ZI^)^\RA:J2R^>I {Ys; Ws\Wa\s, Wb, Zx,) ,V nonempty set 5 C ^ I 
I les J 

where Zi -^ Wi -^ Yi ^ {X, Yxj^\^iy, Wxj^\^iy, Zxi^\{i]) form a Markov chain for all i G II- 

It's easy to check that TZ{Wa\Wb,Zxl) is a contra-polymatroid with \A\\ vertices. Specifically, if tt is a 



permutation on A, define the vector i?^(7r) by 

-RttwW = I{Y^{t)]W^(^^)\W^(^i+i),--- ,W^qa\),Wb,ZiJ, i = I,--- ,1^1 -1, (9) 

K(\A\)i^) = I{y.{\A\);W^(\A\)\WB,Zi,). (10) 

Then -R^(7r) is a vertex of TZ{Wa\Wi3, Zjj^) for every permutation tt. The dominant face 'D{Wj(\Wb, Zx^) of 
'R-{Wj\]WbtZj^) is the convex polytope consisting of all points R^ G "^{WaIW^, Zj^) such that J^ieA^i ~ 
HYa; Wa\Wb, Zi^). We have dim [V{Wa\Wb, Zj^)] < \A\ - 1, where dim [V{Wa\Wb, Zj^)] is the dimension 
of'D{Wj\\Z). The equality holds only when the |^|! vertices are all distinct. Any rate tuple i?_4 e 2?(W^|Wb, Z-j^) 
has the property that 

R'a < Ra ^R'a = Ra, yR'A e n(WA\WB, ZiJ. 

Theorem 2.1: For any rate tuple R^ G 'D{Wj(\Wbi Zj^), there exist random variables (W7i, • • • , W/„.)ig^ 
jointly distributed with (X, Ij^^ , Wx^, , Zx,^ ) satisfying 
(i) {Wlj^.)i£_A = WOi (i.e., {Wi^.)ieA and T4^_4 are just two different names of the same random vector), 
(ii) J2teA ^i ^ l-^l + dim [V{Wa\Wb, Zi^)\ and m, < 2 for all i e A, 
(iii) Z, ^ Wl^ -> Wi'„^ -^ r, -> (X,rx^y{,},Tyi^\{,},Zx^\{,},M^;2:„^,j G ^\{i}) form a Markov chain for 

all i e II, 
and a well-ordered permutation a on {W^j , i £ -^} such that 

rrii 

R,=Y.l{Yf,w:j{Wl^}-,WB,Zi,), Vie A. (11) 

Proof: The theorem can be proved in a similar manner as in [12]. The details are omitted. ■ 

When A — Tl and B — (/), Theorem l2.1l savs that if Zi^^ is available at the decoder, then encoders 1, 2, • • • , L can 

convey Wj^^ to the decoder via a (2i — l)-successive Wyner-Ziv coding scheme as long as Rjj^ G TZ{Wxl \Zij^)- 
It is noteworthy that 2L — 1 is just an upper bound, for the rate tuple on the boundary of T>{Wx^ \Zxl), the 

coding complexity can be further reduced. For example, consider the case where L = i. Let Vi be the vertex 

corresponding to permutation tti = (1, 2, 3), i.e., 

Vi = [I{Yi-Wi\Zx,,W2.W:i)J{Y2;W2\Zx,,W^),I{Y^;W:i\Zx,)). 
Let V2 be the vertex corresponding to permutation 112 = (1, 3, 2), i.e., 

V2 = iI(Yi;Wi\Zx,,W2,W3), I{Y2; W2\Zx,), I{Yr, Wz\Zx,,W2)). 

For any rate tuple Rx^ on the edge connecting Vi and V2, we have i?i ~ IiYi\ WilZx^, W2, W3). Hence encoder 
1 can use a Wyner-Ziv code to convey Wi to the decoder if (Zi3,W2, W3) are already available at the decoder. 
Since {R2, R3) is on the dominant face of TZ{W2, W3IZ13), by Theorem 12. II decoder 2 and encoder 3 can convey 
(W2,W3) to the decoder via a 3-successive Wyner-Ziv coding scheme if Z13 is available to the decoder Thus 
overall it is a 4-successive Wyner-Ziv coding scheme. 



In general we can imitate the approach in [26]. For C ^ C II, define the hyperplane 



n{A) - I 7^I, e 7^^ : ;^ R, = /(lU; Wa\Zi,) \ 



and let JT^ = TL{A) n 'D{Wi^\Zx^). If C .4i C ^2 C • • • C ^A; C II, is a telescopic sequence of subsets, 
then Tai H J-A2 H • • ■ fl T^k is a face of 'V{Wx^ \Zxl)- Conversely, every face of 2?(Wxj, \Zi^) can be written in 
this form. Let Bi = Ai\Ai-i, i ~ 1, 2, • • • , /c + 1, where we set Ao — ^ and Ak+i — T-l- Let S be the set of 
permutation tt on Xl such that 

Each permutation tt G S is associated with a vertex of Tax ^ ^Ai H ■ • • fl Ta^ and vice versa. Hence Tax ^ ^A^ ri 
• ■ • n J-Ak has totally |S| = Ili^i (I'^ilO vertices. Moreover, we have dim{J^Ai ri J'A2 H • ■ • n Ta^) < L — k ~ 1, 
where the equality holds if these |S| vertices are all distinct. For any rate tuple i?Xt G ^Ai ri ^A2 H • • • n J-Af,, 
it is easy to verify that 7?.g. is on the dominant face of TZ{WBi |W^ I'-i g i Zj^), i = 1,2, ••• ,k + l. Hence by 




successively applying Theorem 12.11 we can conclude that an (L + L)-successive Wyner-Ziv coding scheme is 

sufficient for conveying Wi^^ to the decoder if it has the side information Zj^ , where 

fc+i 
L = Y^ dim V (Wb, \ Wy .-1 ^^ , ZjA = dim{TAx ^ ^A^ n ■ • • n Ta^ ) ■ 

1=1 

Corollary 2.1: Any rate tuple _Ri^ on the dominant face of TZ{Wx^ ) can be achieved via a iC-successive Wyner- 
Ziv coding scheme for some K < 2L — 1. 

Proof: Apply Theorem 1 with Zj^ being a constant. ■ 

This successive Wyner-Ziv coding scheme has a dual in the multiple access communication, which we call the 
successive superposition coding scheme. 

Consider an i-user discrete memoryless multiple-access channel. This is defined in terms of a stochastic matrix 

W :X X---X XL^y 

with entries W{y\xi, • ■ • , xl) describing the probability that the channel output is y when the inputs are xi, ■ ■ ■ , xl- 
Now we give a brief description of the successive superposition coding scheme. 

Let Xi,x„j , -'^2,x„2 , • • • , Xl,i„,^ be independent, i.e., 

p(a;i,r„i , a;2,i,„2 , • ■ • , ^l.i^^ ) = p{xi^x^^ )p{x2,i^,^ )■■■ p{xl.i^^), 

and Xii-a- G Xi for all i G I^. Let <t be a well-ordered permutation on the set {Xix^ , -^2.1™ ,■■■ , Xl,i„^ }• 

Encoder i: Let n-vectors Xi.i(l), • • • , X^ i(Mi 1) be drawn independently according to the marginal distribution 
p(xi_i), where Mi^i — [2"'^'^'^''i'^l'f'^''i'''' ^^^'i^]. Successively from j = 2, j = 3, • ■ • , to j' = m^, for each vector 
(fci,--- ,fcj_i)withfc^ G {1,2,--- ,M,,4 (s = I,--- ,j-l),letXjj(fci,--- ,fcj_i,l),--- ,Xij(fci,--- ,%_i,Afij) 
be drawn i.i.d. according to the marginal conditional distribution p(xij|a;i,i, • ■ • , Xij-i), conditioned on Xi.i(fci), • ■ • , 



Xjj-i(fci,--- ,fcj-i). Here M,j = |-2"(^(^i.i;'*'l{^i.i}<. )-<^'.j)]. Only x^,™^ (A:i , • ■ • , fc^J's will be transmitted. 
Hence the resulting rate for encoder i is 



1 / ^^^^ \ "^^ 






Decoder: Suppose Xi^miikl, ■ ■ ■ ,k^i.) is transmitted, which generates y e 3^" at the channel output. Decoder 
first finds a fci5i(i),s2(i) ^'^'-^ that y and ^si{i),s2{i){ksi{i).s2{i)) are jointly typical. If there is no or more than one 
such fcsi(i),s2(i)' declare a decoding failure. Otherwise proceed as follows: 

Successively from j ~ 2,j ~ 3,- ■ ■ to j = J2i=i "^i' if there exists a unique fcsi(j"),s2(j) ^^^'^h that 

decode Xsj(j),s2(i)(^si(j),i' ^siO).2, • • ■ , ^si(j),s2(j))' otherwise declare a decoding failure. 

By the standard technique, it can be shown that Pr(kij = k* j,\/i eTl,j E 2^.) ^ 1 as n -^ oo. 

It is easy to see that if we let X- j ~ X.i^ (Vi E Tl,J G 2r„j.), and replace Xij by X'^ ■ in J12L i?^ is unaffected. 
Hence there is no loss of generality to assume X i — ^ X,2 ^ • • • ^ X.mi -^ {Y,Xj^2^,,j E TL\{i}) for all 
i EIl- With this Markov structure, this scheme can be understood more intuitively since we can think that along 
this Markov chain, high rate codebook is successively generated via superposition on low rate codebook. We refer 
to the above coding scheme as X)j=i '7^i -successively superposition coding. 

Our successive superposition coding scheme is similar to the rate-splitting scheme introduced in [12]. Actually 
every rate-splitting scheme can be converted into a successive superposition scheme. To see this, for each user 
j, let fi be a splitting function such that Xi = fi{Ui^i,Ui.2,- ■ ■ ,Ui^,n,) and let X,™, = Xi,Xij = Ui^j^J = 
1, 2, • • • ,771^ — 1, i E II- Then X_i -^ Xi_2 -^ ■ ■ ■ ^ X,„i. -^ (Y, Xjj^ . , j G ^i\{*}) form a Markov chain 
for all i E Tl- In [12] C/i,i, C/i,2, ■ ■ ■ , Ui^mt are required to be independent^, if we remove this condition, then 
every successive superposition coding scheme can also be converted into a rate-splitting scheme by simply setting 

UiJ — Xij,yj E Irrii and fi{Ui^i, Ui,2, ■ ■ ■ , Ui^mJ = Ui^rrii- 

Let 

7^(XIJ - J i?i^ e M^ : ^ i?, < / {Xa; Y\Xi^\a) ,V nonempty set ^ C Z^ i . 
I leA J 

Ahlswede [27] and Liao [28] proved that 

C^conv y 7^(XIJ 

\pixi)p(x2)---pixL) 

where C is the capacity region of the synchronous channel. 

'This independence condition is unnecessary since !7i,ii f^!,2i ■ ■ ■ ,Ui^m- are all controlled by user i. But this condition facilitates the 
codebook construction and storage since now the high-rate codebook at each user is essentially a product of low-rate codebooks. 
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It can be shown that if p(xi, X2 ■ ■ ■ , xl) ~ p(xi)p{x2) ■ ■ ■p{xl), then TZ{Xx^) is a polymatroid [6][7] with LI 
vertices. Specifically, if tt is a permutation on Tl, define the vector Rj^{tt) by 



,L-1, 



(13) 
(14) 



Then Rj^ (it) is a vertex of TZ(Xx,^ ) for every permutation w. The dominant face of TZ{Xxj^ ) is the convex polytope 
consisting of all points Rj^ G TZ{Xx^) such that X]j=i ^i — I{Xz^:Y). Any rate tuple Ri^^ on the dominant 
face of TZ{Xxj^) has the property that 

R'l, > Rx, => R'l, = Rx, , Vi?i^ e n{XiJ. 



The following corollary is a dual result of Corollary 12.11 The proof is similar to that of Corollary 12.11 and thus 
omitted. 

Corollary 2.2: Any rate tuple Rx^ on the dominant face of TZ{Il) can be achieved via a J-iT-successive super- 
position coding scheme for some K <2L — 1. 

Although we assumed discrete-alphabet sources and bounded distortion measure in the previous discussion, all 
our results can be extended to the Gaussian case with squared distortion measure along the lines of [29]-[31]. 
Now we proceed to study the quadratic Gaussian CEO problem [32], for which some stronger conclusions can be 
drawn. Let {X(t)}^i are i.i.d. Gaussian random variables with zero mean and variance a\. Let {Yi{t)}'^i = 
{X{t) + Ni{t)}^i for all i G II, where {Ni{t)}^i are i.i.d Gaussian random variables independent of {X{t)}'^^ 
with mean zero and variance al^, . Also, the random processes {Nk{t)}^i and {Nj{t)}^^ are independent for 
j ^ k. For each i e Xl, let Wi = Yi + Ti. where Ti ^ Af{0, a^. ) is independent of {X, Yj^ , T2^\{i})- Moreover, 
let 



1 



IiY„W,\X) = -log 2 



— , VzGJl. 



'Ni 



(15) 



It was computed in [22], [34] that 

n{Wx,) = 



i?x, : ^ i?« > 2 log 



1 v^ l-cxp(-2r,) \ 
^+.^. ^. 



L 

Y. 



1 I Y^ l-exp(-2ri) 



y Ti , V nonempty set A (=Xl 



ieA 



= 7^(rxJ. 
Furthermore, it was shown in [33], [34] that 



TZiD)= U 7^(rxJ, 



where 



T{D) = \rj,eRl:^+Y^ 



1 - cxp(-2r,) ^ 1 



k ^ 1=1 

Definition 2.4: Let dTZ{D) denote the boundary of TZ{D), i.e.. 



'JV. 



D 



} (16) 
(17) 

(18) 

(19) 



dniD) = {Rx^ e n{D) : R'x^ < Rx^ ^ R'x^ = i?!^, for all R'x^ £ T^{D)}. 
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Clearly, any rate tuple inside TZ{D) is dominated by some rate tuple in dTZ{D). Therefore there is no loss of 
generality to focus on dTZ{D). 

Now we proceed to compute dTZ{D) for the quadratic Gaussian CEO problem. The closed-form expression of 
dTZ{D) is hard to get. Instead, we shall characterize the supporting hyperplanes of TZ{D), since the upper envelope 
of their union is exactly dTZ{D). The supporting hyperplanes of '^{D) have the following parametric form: 



y^ oiiRi = (/'("It 



(20) 



j=i 



where ax^ is a unit (Z2-norm) vector in R;^ and 



<p(aii,) = min 'S^ UiRi. 

^ I— 1 

Since 7Z{rxL) is a contra-polymatroid, by [7, Lemma 3.3], a solution to the optimization problem 



(21) 



miny^ a iRi subject to Rjj^ G 7?.(ri^) 



(22) 



is attained at a vertex i?ij^(7r*) where is n* any permutation such that a^.fi) > • • ■ > a^^'^L)- That is, 

L L L-l / i \ L 

min ^aji?,, = ^ aji?j(7r*) = ^ | (a^.(i) - a^.(i+i)) ^ i?^.(j)(7r*) + a^.(L) ^ i?^.(i)(7 



Ra^enirxJ ^^^ 



Hence we have 



i=l 



i=l 



i=l 



L-l 

= ^ («7r*(i) — a7r*(4+l)) 
1=1 



/ 1 



2 ^ 






+a,r*(L) 



ilog 1 + ai^ 



1 I Y^ l-exp(-2r„..(j)) 

\ <?^ ^ ^T 

\ ^ J=i+1 "t*(j) 



1— exp( — 2rj) \ 






1 — cxp(— 2rj 



'AT, 






L-l 



<p(a2:i,) = min^ ^ (a7r'(j) - a7r*(i+i) 



- loa; 
2 ^ 



/ 1 _i_ Y^ l-cxp(-2rj) \ 

^ ^ — s^:^ — 



V 






l-cxp(-2r^..(j)) 






subject to 



+a..(L)|ilog(l + aiE-"'''^"''^^ 



j=i 



'TV, 



E^ 



^ + E 



1 - exp(-2r,) J_ 



(23) 



(24) 



Since we can decrease rTr«(i) to make the constraint in ( I24t tight and keep the sum in ( I23t decreasing at the same 
time (If r7r->(i) attains but the constraint in ( I24t is still not tight, then apply the same procedure to 7'7r*(2) and so 
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on.), we can rewrite ( I23t and ( I24t as 



L-l 






'•^1.^'^+ ^=l 



2 \ (Tt' "^ — ' CTaf 



+«-*wUiogil+E^. 



j=i 



subject to 



1 ^ 



1 - cxp(-2r,) _ 1 



(25) 



(26) 



Let Tj be the minimizer of the above optimization problem. Introduce Lagrange muhipUers Aj^ G K^ for the 
inequality constraints rx^ E M;^ and a multiplier j/ G M for the equality constraint ( I26> . Define 

L-l 

Glr-ii , Ai^ , I/) = E (a7r*(i) -"^*(i+i)) 



V=i \ ^ j=»+i 

.'(.) iiogf+E'^. -E^^-'--(i+E 



D-i:)cxp(-2r^.(j)) 



'Af, 



*(j) 



We obtain the Karush-Kuhn-Tucker conditions [35] 

rl > 0, Afc>0, Xkrl^O, fc = 1, 2, • • • , L, 



1 - exp(-2ri) 



(27) 



'w. 



'X 



1 - exp(-2r*) _ ^ 



fc = l 



^W. 



ac 



l9?'7r-(l) 

dG 



2!yexp(-2r;w^J 



5r, 



Tr'(fc) 



'•'^*(i)=''. 



r„-(k}=r^ 



*(i) 



*(fc) 



'W, 



+ Q^7i-(1) ^ ^7r*(i) — 0, 



exp(-2r:.(,)) ^1^ J 1 ^ l_exp(-2r:.(,,) 



'A^^ 



'(fc) 1=1 



7 = 1 \ ^ l" = »"+1 



i = i+l 



'W^ 



+"71* (fe) — K*{k) 



2i.exp(-2r* 



(fc)^ 



'JV^. 



(fc) 



exp(-2r;. 



{k)) 



fc-i 



'JV^. 



^(a^.(,) - a^.(i+i)) n - E 



1 v^ 1 - exp(-2r;. 



U)' 



(fc) i=l 



J = l 



'N,', 



2z.exp(-2r;.(^.) 



V*(fe) 



•(fc) 



'JV^ 



= 0, fc = 2,3,---,L. 



By the complementary slackness condition, i.e., A^ > ^ r^ — 0, we can solve these equations to get 

+ 



W-ii) 



' -K'ik) 



1, 2v 

-log 2 

2 a^.(i)cr^^ 



(28) 



^log 



fe-i 

1=1 



2z/ + X] ("7r*(i) - a7r*(i+l)) \TS - Y. 



l-cxp(~2r-;,(j)) 
j = l ''"^-O) , 



-'\ 



,fc = 2,3,---,L(29) 
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where v is uniquely determined by the distortion constraint 



o-y ^-^ air D 

and Tj can be computed recursively from t-*,^-,, r'^,(2), • • • j to r*w^N. 

In the above we assume a^ > for all i e T^. Now suppose a7r»(i) > ■ • ■ > ct^^trZ) > = a^./2:+i~i = • ■ ■ = 
a^.(i). We can let r-;.^-^^^ = • . . = r;.(^) = oo. If 

J_ A 1 l_ 

then we have '''*.(i) = • • • = ''*,., ^ = and correspondingly ip{ax^) = 0. Otherwise, solve r^.fi), • ■ ■ , ' ,,7:^ 
from (I28> ( I29> with the distortion constraint (13 0> replaced by 



(31) 



. r 



^2+2^ ^2 + Z^ ^2 



^^E - ^l-'-'^ -^^^i- 



Let T{aij^, D) with a,; > (Vi G Xl) be a supporting hyperplane of TZ{D). By (I18> . we have 

T{ax, , i?) n a7e(i?) = r(ai, , i?) n IZiD) = T{aj, ,D)nl U TZin, ) 

If T{ax^,D) n 7^(^I^) 7^ for some rj^ e ^(-D), then we must have Ri^{-k*) G T{aj^,D) D n{rx^), 
where Rxj^{tt*) is a vertex (associated with permutation tt*) of TZ{rx^)- Now it follows by the above Lagrangian 
optimization that TZ{rxj^) = ^('ij^)- Therefore, we have 

T{ax, , D) n dn{D) = T{ax, , D) n 7^(4J. 

Clearly, T{axj^ , £*) n 7^(?'j ) is a face of the dominant face of TZ{r^ ). Let {B[ , • • • , i3^.) be a partition of Z^ such 
that am = a„ for any «„, a„ G S,' (i = 1, 2, • • • , /c) and «„ > a„ for any a™ G S^, «„ G B'j {i < j). Let S' be 
the set of permutation tt on Z^ such that 

^(EI^^I + 1 '•••'^(EI^^I [=^" z = l,2,...,fc. 

T(axj,, D)nTZ{rj ) has totally |S'| = Ili^id'^j'IO vertices, each of which is associated with a permutation vr G S'. 
Furthermore, dira{T{ax,^,D) n 7?.(rj )) < L ~ k, where the equality holds if these |S'| vertices are all distinct. 
Finally, we want to point out that if ai = • ■ • = a^, then T{axj^^ D) r\TZ{rj ) is the minimum sum-rate region of 
n{D) [22]. 

Corollary 2.3: For the quadratic Gaussian CEO problem, any rate tuple Rx,^ G dTZ{D) can be achieved via a 
ii'-successive Wyner-Ziv coding scheme for some K < 2L — 1. 

Proof: Since TZ{D) — [J^ eriD) ^(^ii,)' for any rate tuple Rx,^ G dTZ{D), there exists a vector rj^ G J-{D) 
such that Rxj^ G TZ{rxj^)- Furthermore, by Definition 12.41 it's easy to see that Rx,^ must be on the dominant face 
of TZ{rx^)- Now the desired result follows from Corollarv l2.1l 
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To get detailed information about the coding complexity of a rate tuple Rx^ G dTZ(D), we can proceed as follows. 
Let T{ax^ , D) be the supporting hyperplane of dTZ{D) such that Rj,^ G T{axj^ , D) n dTZ{D). Use the Lagrangian 
optimization method to find 7^(^J^) with rj^ G T{D) such that T{axL,D) n dn{D) = T{ax^,D) n 7^(4^). 
Let T C T{axj^ , D) n 7?,(rj ) be the lowest dimensional face of 7?,(rj ) that contains Rx^ ■ We can conclude that 
Rx[^ is achievable via an (L + (ii?7i(jr)) -successive Wyner-Ziv coding scheme. ■ 

III. Distributed Successive Refinement 

In the previous section, we have shown that the successive Wyner-Ziv coding scheme suffices to achieve any 
rate tuple on the boundary of rate region for the quadratic Gaussian CEO problem. We shall extend this result to 
the multistage source coding scenario. 

Definition 3.1: For Rxl,i ^ Ril,2 < • ■ • < Rxl,m and Di > D2 > ■ • ■ > Dm, we say the M-stage source 
coding 

{RilA^Di) / {Rxi,,2, D2) / ■■ ■ / {Rxi,,m,Dm) 

is feasible if for any e > 0, there exists an no such that for n > n^ there exist encoders: 

4? • ^r ^ {1, 2, • • • , L2"(«-^"«---+^)j } , z = 1, 2, • • • , L, J = 1, 2, • • • , M, 

and decoders: 

5^ ^ n n {l' 2' • • • ' L2"(^-'=-^-''-^+^^J }^X^, J = 1, 2, . • • , M, 



such that 



where 



k=li=l 

n 



J2d(x{t),X,{t) 



,t=i 



<Dj+e, j = l,2,---,M, 



X, = g^^ (./}"i^(Yi), • • • , /S(Y^), • • • , /g(Yi), • • • , /S(Yl)) , J = 1, 2, . . . , M. 

Here we assume Rxl.o = (0, • ' ' j 0). 

The following definition can be viewed as a natural generalization of the successive refinement in the single 
source coding [36]-[39] to the distributed source coding scenario. 

Definition 3.2 (Distributed Successive Refinement): Let D*(Rx,^) = inm{D : Rx^ G Tl{D)}. For Rxl,i ^ 
Rxl.2 < • • • < Rxl-M, we say there exists an M-stage distributed successive refinement scheme from Rxl-I to 
Rxl,2, to , to Rxl,m if the M-stage source coding 

{Rx,.i,D*{Rx,.i)) / {Rx,a.D*{Rx,,2)) /■■■ / {Ri,.m.D*{Rx,,m)) 

is feasible. 

Theorem 3.1: For Rxl,i < Ril,2 < • • • < Ril.m and Di > D2 > ■ ■ ■ > Dm, the M-stage source coding 

(RilA^Di) / (i?I^,2,-D2) / ■■ ■ / {Rii^,M,Dm) 
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is feasible if there exist random variables Wx,^ ,1^ jointly distributed with the generic source variables {X, Yjj^ ) 
such that 

where Wx^.Xm satisfy the following properties: 

(i) Wi,i -^ Wi^2 -^ > Wi.M -^Yi^ {X, Yx^\{.,},Wx^\{i},Xm) form a Markov chain for all i G Il\ 

(ii) For each j G Im, there exists a function Xj : 11^=1 ^i-j ~* '^ such that E(i(X, Xj(Wxf, j)) < Dj- 



Proof: By Theorem 12. II we can see that each stage can be realized via a (2L — l)-successive Wyner-Ziv 
scheme. ■ 

The Af -stage source coding, if realized by concatenating M versions of (2i — l)-successive Wyner-Ziv coding 
schemes, is essentially a {2ML — Af )-successive Wyner-Ziv coding scheme. But it is subject to more restricted 
conditions since a general (2ML — A/)-successive Wyner-Ziv scheme (satisfying the rate constraints: Rxl.m and 
the distortion constraint: Dm) may not be decomposable into A/ versions of 2i — 1 successive Wyner-Ziv scheme 
with rate and distortion constraints satisfied at each stage. 

In the remaining part of this section, we shall focus on the quadratic Gaussian CEO problem. 

Lemma 3.1: For Rxl,i < Rxl-2 < • • • < Rx^.m and Di > D2 > ■ ■ ■ > Dm, the Af-stage source coding 

{Rx,A,Di) / {Rx,,2,D2) /■■■ / (Rx^auDm) 

is feasible if there exist ri^^.j G K;^, j = 1, 2, • • ■ , A/, satisfying 
(i) ri^j-i < rx^j for all j G Xm, 

(ii) l/aj, + Y.ti (1 - exp(-2r,j)) /aj,^ ^ l/D, for all j G Xm, 
such that 

}^(i?,,-i?,,_i) > -log---logl-^ + }_^ -^ +^ ^ I 

+ 2j(^ij — rjj_i), Vj G Xa/ , V nonempty set A C I^. (33) 

ieA 

Here we assume rx^.o = (0, • • • , 0). 

Proof: Let W,,m = >"»+T,,m and W,j = W,j+i+T,j (j G Im-i), where r,,^ ~ J\f{0, a^^J, i G Xl, j G Xm 
are all independent and they are also independent of {X, Yx^). Let 

M 

r,,, = /(K,; W,, |X) = - log j^ (34) 

k=j 



and E(X - ¥.{X\Wxr^,j)y = -Dj for all j G Xa/ , i.e., 

^ 1 - exp[-2rij) _ 

~ D 



1 ^^ l-exp(-2...) 1_ ,33, 



'X ,-_i "N, 
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Let Rjj^fi = (0, • • ■ ,0) and let Wx^fi be a constant vector. By Theorem l3. II for any Rx^s < Ril,2 < • • ■ < 
Ril,m with 

(Ri^j ~ Ri^,j-i) e niWx^jWx^,j-i), Vj e Im, (36) 

the 7\f -stage source coding 

{Ri,.i,Di) / {Rx,.2,D2) /■■■ / {Rx,.auDm) 
is feasible. We can compute ( I36> explicitly as follows: 

Y,iR^,3 - R^.J-l) > I{yA;WA,j\Wx,\A,j,WA.j-l) (37) 

= I{X,Ya;Wa.j\Wx,\a,j,Wa.3'i) (38) 

= /(X; W^a,|W^i,\Aj-. Wa.3-i) + HYa; Wa,j\X) - I{Ya; Wa,j-i\X) (39) 

= h{X\Wx,\A.,,WA.j-i) - h{X\Wx,,,) + ^(r,,, - r,,,_i) (40) 

ie-4 

1 1 1 I ,-^ 1 -cxp(-2ri.j_i) ,-^ l-cxp(-2rjj) 
- 2^°gl3--2l°M^+^ ^^^"^ ^ < 

+ yj(rij — rij_i), V nonempty set ^ C I^ . (41) 



ieA 



ieA 

The proof is now complete. 
Lemma 3.2 ([31], Lemma 1): 



1^'" f(") ^^ii.„^ 



-/(X;/^;'^^^.)>-log-i, VjelM. (42) 



p(") I'w,- ^ /i^ o„H r'. - f(") 



The next lemma is a direct application of [34, Lemma 3.2] with d = f^j {\fi G A) and d — /^ j (Vi £ Il\„4), 
where /.^J is the abbreviation of (//"'(Y,), ■ • ■ , f^^\Y,)) . 

Lemma 3.3: Let r,^ = i/(Y; /[!^^. |X), Vi £ Xl, Vj £ Iaj. We have, for all < j < A: < M, 



where />q (i £ Zl) are constant functions and rj^ q = (0, • • ■ ,0). 

Lemma 3.4: For i?jj^,i < Rxl-2 < • • ■ < Ril-I^i ^^d -Di > -D2 > • • ■ > ^'a/, if the M-stage source coding 

{Rx,A,Di) / {Rx,,2,D2) /■■■ / (Rx^auDm) 
is feasible, then there exist rx^j £ M.^, j = 1, 2, • • • , Af, satisfying 
(i) ^it j-i < rx^j for all j £ Im, 

(ii) l/a| + Ef=i (1 - cxp(-2r,.j)) /<^ > 1/D, for all j £ Im, 
such that 

l-exp(-2rij) ^ .^^ 1 - exp(-2ri,fe) 






E 



'^. ,:eia^ ^• 



E^'^^'^ ~ ''«.■?)' V < j < A: < Af, V nonempty set ^ C X^. (44) 



ieA 



Hereri^,o = (0,---,0). 

Proof: Let r,j = I{Y; f\'^^\X)/n {Vi G Xl,Vj G Im)- It is clear that rj^j-i < rj^j for all j e Xm- 
Moreover, if we let A = II in ( I43t . we have 

1 , V^ 1 - cxp(-r,,fe) 1 1^2 („) A, 1 w;^T 

"x j^i "Wi "x V" / ^fc 

where the last inequality follows from Lemma 13.21 
Furthermore, we have 

k 



YJR^M-R^,) > ^T.T. Hi&>lH{f^ll,-s=J + lr--,k) 



n 
ieA ieAs=j+i 






> 






10^7^^ log -^+E ^'^^:^"" + E 



\ X ,cA Ni ielL\A ^i 






\^ "^. / ^eA 



Denote this rj^ by rj^ [Ril)- We have 



-X ,^1 "X. 

and 



1 >^ l-cxp(-2r*(i?xj) ^ 1 



(45) 



(46) 



> -^(Y^;A:i'^=j'+i'---'^i/x:w.'/Xx,) (47) 

- ^/(X,Y^;/i(:l,,s=j + l,...,fc|/(';)_4^^^,/i%) (48) 

+ ^ E ^(Y.; /S\ .s = J + 1, • • • , fc|X, /ij) (49) 



1 1 1 [1 Y^ 1 -exp(-2ri,j) -^ 1 - cxp(-2r,,fc) 



+ ^{n^k~n^j). (51) 

where (15 U follows from Lemma IT^ and Lemma l?31 Now the proof is complete. ■ 

Lemma 3.5: For any Rx^ G M;^, there exists a unique rj^ G M.^ satisfying 

L 



_^^^ l-exp(-2.0 ^ 1^ (52) 

and for any nonempty set A '^ II 

1 — exp(— 2ri 



E^- (53) 



(54) 



E^^-^iogi^ + E-r(^-)- (55) 

Proof: See Appendix. ■ 

Now we are ready to prove the main theorem of this section. 



Theorem 3.2: For Ri^.i < Rxl,2 < ■ • • < Ril.m, there exists an M-stage distributed successive refinement 
scheme from i?i^,i to Rx,^^2, to , to Ri^^m if and only if 



^{Rtj - Ri,]-i) 



1, 1 

> olog 



1 Jj^^^l_eM^2rpR^^^^^ 




2 '=D*(i?x.,, 

+ ^{r*{Ri^,j) - r*(i?i^j_i)), Vj e XAf,V nonempty set ACI^ 

ieA 

Here i?i, ,0 = 4 Ji?i, , 0) = (0, • • • , 0). 

Proof: Let Z3j = D*{Rx^.j) (Vj G X^/ ) in Lemma l34l Suppose the vector sequence rj^j {j — 1,2,- ■ ■ , M) 
satisfies all the constraints in Lemma 13741 By Lemma l331 we must have l/(j\ + X]i=i(l ~ 6'^P(~2''i.j))/o'Ar = 
\/ D*{Rj^,j). So the constraints in Lemma l34l implv the conditions in Lemma 3.L Therefore, the conditions in 
Lemma im are necessary and sufficient. Furthermore, by Lemma l331 rTj j. if exists, must be equal to ? j (Ri^.j)- 
The proof is thus complete. ■ 

Remark: Applying ( I55t and then ( I54> . we get 

tiR.. - R..-.) = ^l"g^^^ +E^*(^x..) -1^0, ^^^^ trUR,^.,^,) (57) 

L 

+ E (r*(i?i.J - r:(i?x„,-i)) , Vj e Im- (58) 

i=l 

Hence in ( l56l the constraints on X]i=i(^ij ~ Rij-i):j == I7 2, • • ■ , M, are tight. 

The sequential structure of ( I56> leads straightforwardly to the following result. 

Corollary 3.1: For i?i^,i < i?Xj^,2 < • • • < Ril,m, there exists an Af -stage distributed successive refinement 

scheme from Ri^^i to Rx^,2, to , to Rx^^m if and only if there exist a sequence of 2-stage distributed 

successive refinement schemes from Rx^j^i to Rx^.j, j = 1,2,- ■ ■ , M. 

Corollary 13.11 shows that for the quadratic Gaussian CEO problem, we only need to focus on 2-stage distributed 
successive refinement. 

By ( I34> . each monotone increasing vector sequence rj^j (j = 1, 2, • • • , M) is associated with a unique a^ 
(j = 1, 2, • ■ • , Af ) and thus a unique M^j^ ^ (j = 1, 2, • ■ • , A/). We shall let W|^ {Rtl,j) denote the Wx^^.j that is 
associated with rj [Rxl-j) (J = 1; 2, • ■ • , Af ). Now we state Theorem l3.2l in the following equivalent form, which 
highlights the underlying the geometric structure. 

Corollary 3.2: For Rxl,i ^ Ril-^ < ■ ■ • < Ril-M, there exists an Af -stage distributed successive refinement 

scheme from Rx^,! to Rxl,2, to , to Rxl,m if and only if {Rxl.j^Ril.j-i) ^ 2)(WJ^ (^li j)|W|^ {Ril,]-i)), 

where 'D{W^^{Rx^,j)\W^^{Rx^,j-i)) is the the dominant face of 7^(W|^ (i?i^ j)|WJ^(i?i^ j_i)), Vj G Xm- 
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Proof: It is easy to verify that ( I56> is equivalent to 

Y,{R^,J - R^^J-l) > I{Yx, W\{Ri^^j)\WX.{Ri^^j), W^l(i?i^j-i)) , Vj e Jm,V nonempty set A C I^, (59) 
which, by Definition 12.31 is equivalent to 

(i?x.,, - i?x.,,-i) e 7^(M/:^^ {Rx^.jWi^ {Rt,,j-i)), Vj e Zm. (60) 

Furthermore, J58> is equivalent to 

L 

which means -Rij^ j — Rx^j-i is on the dominant face of TZ{Wj {RxL-j)\^i [Rxl-j-i))^ ^i ^ '^m- • 

Remark: Let Ti be the lowest dimensional face of T>{Wj (-Rji j)|WJ {RxlJ-i)) that contains -Ri^ j — Rx^j-i- 
By the discussion in the preceding section, we can see that this i\/-stage distributed successive refinement can be 
realized via an {ML + J2i=i dim{!Fj))-s\iccessive Wyner-Ziv coding scheme. 

Now we proceed to compute rj (Rxl)- It is easy to see that rj [Rxl) i^ the maximizer to the following 
optimization problem: 



1 ^ 1 - oxp(-2r.) (^2) 



max „ 



subject to 



/ 



^log 



1 y^ l-cxp(-2r,) \ 

^ ^. ^IT 



1— exp( — 2ri) 



E ^^ 



yj Ti < yj i?,;, V nonempty set A cIl, (63) 



ieA i£A 



and 



\ ^ 1=1 ™i / i = l 4 = 1 

which is essentially to find the contra-polymatroid TZ{rj^) that contains Rx^ and has the minimum achievable 
distortion D{rxj^) = l/c^ + Sj=i(l ^ exp(— 2ri))/(T^.. Another approach is use the Lagrangian formulation in 
the previous section. That is, first characterize r^ {Rxl) for i?i^ G dTZ{D) via studying the supporting hyperplanes 
of dTZ{D) for fixed D. Then change D to get rj (Rx^ ) for ^H ^Xt ■ This approach is in general more cumbersome 
than the first one. But for small L, it is relatively easy to get the parametric expression of r J {Rx^ ) via the second 
approach. 

To give a concrete example of the distributed successive refinement, we choose to study the special case where 
L = 2. We shall adopt the second approach. It is easy to see that i?!^ is either a vertex of TZ{rl{Rx2),r2{Rx2)) 
or an interior point of the dominant face (which is a line segment) of TZ{rl{Rx2),r2{Rx2))- For the first case, 
(r*(i?i2), ?'2(i?i2)) is completely determined. For the second case, i?!^ must be on the minimum sum-rate line of 
dTZ{D* {RX2)) ■ Hence we only need to study one supporting line of dTZ{D), namely, R1+R2 = min(fl;j ;j,,)gaK(D)(^i" 
R2), which has been characterized for all D in [22]. 



Without loss of generality, we assume a% < f^ . Let 



where 



1 _ 1 A 1 



i=l 



Let D be the unique solution to the following equation: 

Ld 



Let 



We have 
(i) If 

then 



(ii) If 



then 






Nt \ D„i„(LD) D 



^Wi (d„.„\l5) ^ j , 



^5 = 1: 



iV2 V X "^JVi N2 



D 



iiog(-L^i^£Mz^Uiw.2 



(iii) Otherwise r*{Rx^) ~ ri,i ^ 1,2. 
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I fell 

Ld = max -^ A: e Z2 : ^- + — - — -— > ^ , (65) 

1 <, ^ £'mi„(fc) J 



(66) 



= i?i+i?2. (67) 



n ^ ^l°g -^^^ ^1' (68) 



r2 = { / / x-i\ (69) 



1 / 1 l-exp(-2fi)\ 1 , 

Z \0'x '^Ni / ^ 



o^og ^- + 2 + log '^x+ ''1(^2:2) = -Rl, (71) 

ilogfl + .i^l^^^^Pi^^l&ll')+r*(i?xJ + r*(i?xJ = i?i+i?2. (72) 



^2 > ;^ log ( 3^ + ^ ) + - logaj. + r2, (73) 



ilogfl + ^i^l^^^^Pi^^i&ll^+r*(i?iJ + r*(i?xJ==i?i+i?2. (75) 
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The above three conditions essentially divide R^ into 3 regions. Define 



fll = 


|(i?i,i?2)eK+:i?i>^log 


^ 1 


^3 = 


|(i?i,i?2)eK+:i?2>^log 


' i 
\^x 


^3 = 


|(i?i,i?2)eM+:i?. <^log 


■ i 



^Wl 



■"5 



-logai+riL (76) 



1 — exp(— 2ri)\ 1 2 

■logdl.+fa^ (77) 



l-exp(-2?'2)\ , 1 

\oga\+7,,i = l,2\. (78) 



l-cxp(-2rj)\ 1 2 



It is easy to check that except the boundaries (i.e., those rate tuples that satisfy ( I70> or (I70> with equality), J7i, i72 
and 5^3 do not overlap. Typical shapes of J7i, ^^2 and Vt^ are plotted in Fig. 2. Any rate pair Rx.^ E {fli U il2} is 
a vertex of Tl{rl{Rj^),r2{R-i2)) and thus is associated with a 2-successive Wyner-Ziv coding scheme. Any rate 
pair i?i2 strictly inside ils is an interior point of the dominant face of 7?.(r5^(i?i2)7 ^2(^X2)) ^^^ '^hus is associated 
with a 3-successive Wyenr-Ziv coding scheme. Hence there is a clear distinction between {0,1^0.2) and 1^3. We 
will see that this difference manifests itself in the behavior of distributed successive refinement. 

Henceforth we shall assume i?i2,2 > Ri2,i- 

Claim 3.1: {rl{Ri,a),rl{RT,a)) > {rl{Ri.,i).r*2{Ri2,i))- 

Proof: If both Ri^.i and i?i2,2 are in f2i or both Ri^^i and Rj:^^2 are in 0,2, the claim can be easily verified 
by checking the equations (17 1> . ( I72> . ( 17 4> and (I75> . Since ?i and r2 are monotone increasing functions of i?i +-R2, 
the claim is also true when both Ri^.i and R12.2 are in Q,-^. 

Now consider the general case when Ri^.i and R12.2 are in different regions, say Rx^^i G i^i and -Ri2,2 G ^3- 
Suppose the line segment that connects -Rj^^i and R12.2 intersects the boundary of i^i and 5I3 at point R'^ . We have 
{rl{R'x,),rl{R'^^)) > {rl{Rx,,i),r*2{Rx,,i)) since both i?i,^i and R'^^ ai-e in fli and {rl{Rx, ,2) , r^Rx, ,2)) > 
{rl{R'x,),r*2{R'x^)) since both R'^^ and ii'i^.s are in ^(3. Hence (r*(i?x,,2), r2*(i?i,,2)) > {rl{Rx,,i),r*2{Rx,s)). 
The other cases can be discussed in a similar way. ■ 

Claim 3.2: If both Rx2,i and Rx2,2 are in ili, then there exists a distributed successive refinement scheme from 
i?i2,i to Rx2,2 if and only if _Ri,2 = -Ri,i or R2.1 — 0. 

Proof: If i?i 2 = -Ri.i, by ( 17 1> we have rj^(i?x2,2) = 7'i(-Ri2.i)- It is easy to verify that the conditions in 
Theorem 13. 21 are all satisfied. If R2.1 = 0, by (I71> and ( I72> . we have ?'2(^i2,i) = 0- Again, it is easy to check that 
the conditions in Theorem 13.21 are all satisfied. 
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Now suppose there exists a distributed successive refinement scheme from Rx2,i to Rx2,2- Since both Rx^,! and 
Rx2,2 are in fli, by dHJ and (|23 



-2 + '^21^2:2,2] ~ TT iOg -2" ^ 72 

' l-exp(-2r*(i?i„i))\ ,^^ ^1, /I l-exp(~2r*(i?i,,i)) 




^ 9 1 ^2(^i2,i) + :^log -2- ' 9 

1 1 li Y 1 , l-cxp(-2r*(i?i,,2)) 



2 ^°^ DiE^ - 2 1°^- 1^ ^ — <; ' ^ ^^*(^^-^) - ^^*^^^-^^ 



2 




l-cxp(-2r*(i?i„i))\ 1, /I , l-cxp(-2rt(i?i„i)) 






By Theorem 13.21 we must have 



1, 1 1, /^ 1 l-cxp(-2r*(i?i,.2))\ ,,„ , ,,„ , 

^ 1 - exp(-2r*(i?i,4))\ 1 ^ /I 1 - exp(-2r*(i?i,,i)) 




■log 



1, , 1 1, n l-cxp(-2r*(i?i„2)) l-exp(-2r*(i?i„i)) 



+r*2{Ri2,2)~r*{Rx2,i), 

which, after some algebraic manipulation, is equivalent to rl{Rx2,2)r2{Rx2,i) — ^1(^X2, i)''2(^X2.i)- Then we 
have either r\{Rx2.2) < 'r\{Rx2,i) (which further implies r\{Rx2,2) — ''1(^12,1)) or r2{Rx2.i) — . Hence, by 
dm and (|22}, we have i?i,2 = Ri,i or i?2,i =0. ■ 

The following claim follows by symmetry. 

Claim 3.3: If both RX2.1 and Rx2,2 are in il2, then there exists a distributed successive refinement scheme from 
Rx2,i to Rx2,2 if and only if i?2,2 — R2,i or i?i,i = 0. 

Remark: Claim 13.21 and 13.31 imply that there exists a distributed successive refinement scheme from Rx2,i to 
Rx2,2 if RX2.1 and RX2.2 are on the i?i-axis or RX2.1 and i?i2.2 are on the i?2-axis. Actually in this case, the 
distributed successive refinement reduces to the conventional successive refinement in the single source coding"* 
[38]. Furthermore, if i?i = cx) and aj^^ = (or i?2 = c« and crj^ — 0), then the quadratic Gaussian CEO problem 
becomes the Wyner-Ziv problem of jointly Gaussian source. Claim 13.21 (or Claim 13. 3> implies the successive 
refinability for the Wyner-Ziv problem of jointly Gaussian sources [40]. 

Claim 3.4: Suppose Rii > 0,i?2.i > 0. Then there is no distributed successive refinement scheme from Rx2,i 
to Rx2,2 if RX2.1 e 0.1, RX2.2 G ^2 or Rx2S e ^2, RX2.2 e ^1- 

Proof: We shall only prove the case for Rx2.i G i^i, ^12,2 G ^2- The other one follows by symmetry. 

^There is a slight difference since the CEO problem, after reduced to the single encoder case, becomes the noisy (single) source coding 
problem. But the generalization of the successive refinement in the single source coding to the noisy (single) source coding is straightforward. 
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By (ED and (|73, i?ia > 0, i?24 > implies rl{Ri^^i) > 0, r*(i?i,,i) > 0, which further implies r*{Ri.^^2) > 
by Claim Ell Now it follows from ^, (|23, (EU and (ES that 



Ri,2 — RlA 






1 , l-cxp(-2rl*(il'I,,l))^^ 1 

2 



exp(-2r2*(i?i,,2)) 



'N2 



;^log(— + ^ ^""^ J'"^""^^^ )-;^loga|-r*(i?x.,i) 



'X 



1, 1 

- log ; 7 

2 ^ D*(Rj,,2) 2 






'X 



' N2 



X 



V^^w, 



1 l-exp(-2ri*(ii"x„i)) 



'Wi 



r*(i?i3,2)-ri*(i?i„i), 



which is strictly less than 



1, 1 

- log 

2 ^D*{Ri,^2) 2 



1, / 1 , l-exp(-2r2*(i?x.,2)) , 1 - cxp(-2rr(i?x„i)) \ , 

iOg ^- H 2 ^ 2 + ^ll^l2,2j 



'X 



'Af2 



'Wi 



'^1(^12 



if rl{Rx2,i) > 0, r2(i?i2,2) > 0. Thus by Theorem 13. 21 the distributed successive refinement scheme can not exist. 




Fig. 2. Distributed successive refinement for tlie quadratic Gaussian CEO problem 



In Fig. 2, the arrows denote the possible directions for the distributed successive refinement in Oi and il^. For 
illustration, we pick a point s in rj2. The dark region is the set of points to which there exists a distributed successive 
refinement scheme from s. We can see that the distributed successive refinement behaves very differently in these 
three regions. 
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IV. Conclusion 

We discussed two closely related problems in distributed source coding: The first one is how to decompose a 
high complexity distributed source code into low complexity codes; The second one is how to construct a high 
rate distributed source code using low rate codes via distributed successive refinement. It turns out that, at least for 
the quadratic Gaussian CEO problem, the successive Wyner-Ziv coding scheme gives the answer to both problems. 
Besides the features (say, low complexity and robustness) we discussed in the paper, the concatenable chain structure 
of the successive Wyner-Ziv coding scheme seems especially attractive in wireless sensor networks, where channels 
are subject to fluctuation. In this case, by properly converting a high-rate distributed source code to a multistage 
code via the successive Wyner-Ziv coding scheme, one can match source rates to the channel rates adaptively. 

Appendix 
Proof of Lemma 3.5 



For any ri^ G M;^, define two set functions /(•, rj^ ),/£>(•, ri^) : 2^^ 



i+. 



/(AriJ - -log 



/ 1 I V^ l-cxp(-2r.) \ 

^ I 7^+ 2^ 7T. 



1 I Y^ l-cxp(-2ri) 



y r.i , V nonempty set A (=Il, 



ieA 



fD{A,rxJ = 2 Sd^2^^\'^^ Z^ 2 + 2^ ri,V nonempty set ^ C J^, 

and/(0,riJ = /z,(0,riJ = O. 

Note that f{-,rx^) is a rank function and induces the contra-polymatroid TZ{rxj^) defined in J17> . Furthermore, 
for any nonempty set A ^Tl, 



HAn.)-lr,iAn.) - 1 log (-^ + g i^iSlpil) - 1 log 1^ (79, 

By the supermodular property of f{-,rxj^) and the equation (I79> . we can establish that, for any rj^ satisfying 
ri > (Vi G Zl) and nonempty sets S,T C Z^, 

(i) 

/(5,rxJ + f{T,ri,) < f{S U T,rj,) + f{S n T^n,)- (80) 

(ii) If l/o-| + E»ii(l - exp(-2n))/cr^^ > 1/D, and 5 ^ T, T ^ S, then 

fD{S,ri,) + fD{T,ri,) < foiS U T, n J + foiS H r,rij. (81) 

It was shown in [34] that 



T^iD) = IJ {Rir. -J^^'- •^^("^' '^2:1, ), V nonempty set ACI^i 
rx, e:F(D) I ieA ) 



(82) 



DRAFT 
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where J-{D) is defined in ( I19l l. Hence there must exist a vector rij^ £ R;^ satisfying the constraints J52t and (15 3> 
in Lemma l331 i.e., 

X! ^» ^ /£>* (flxi ) (-^' ''2:1, ) , V nonempty set ^ C J^ , (83) 



iG-4 



and 



1 A 1 - cxp(-rO 1 ,„,^ 



Let Q = {i ^ II : ri > 0} . Then ( I83> and ( I84> reduce to the following constraints: 



^Ri> fo' (flxi ) {A, rxj, V nonempty set Acg, (85) 



ieA 



and 



^^^ l-cxp(-rO 1^^ (86) 



are still active. Thus without loss of generality, we can assume Q = Tl- 

It can be shown that in J83> . if the constraints on X]iG5 ^« '^^'^ SiGT ^» ^'^^ tight, then either S CT or T C S. 
Otherwise 

fo' (flx^ ) (5, ri^ ) + /r,- (ij^^ ) (T , rj^ ) (87) 

ieSuT ieSnT 

> /z5.(fl,j(5ur,rxj + /^.(j,^j(5nr,rxj, (90) 

contradictory to J81> . Let ^ = Plfcei -^fe' where ylfc(fc g X/f) are the sets for which the constraints on J^ieA ^* 
are tight in ( I83t . If there is no such an Ak, let ^ = Tl- A is thus always nonempty. 
Now suppose 



J_ v^ 1 - exp("ri) ^ 1 



> 7^777^^- (91) 



4 fct ^ir. i^*(i?lj' 

Pick any i* e A, we can decreases r^. to r^. — 5 for some 5 > so that all the constraints in ( I83t and ( I91> become 
non-tight. Then we can decrease D*{Rx,^) to D*{Rj^) — e for some e > without violating any constraints in 
(|83} and ^^. By (|82} we have Rx^ £ 7i{D*{Rj^) - e), contradictory to the definition of D*{Rx^). Hence we 
can conclude that ( I54> holds, i.e.. 



I — exnl — r. I I 

(92) 



J_ .^ 1 - exp(-rj _ 1 



Now we proceed to show that rx,^ must be unique. 

It is easy to check that 1/cr^ + J2i=ii^ ^ exp(— 2ri))/CT^, is a strict concave function of rxj^ and for any 
nonempty set -4 C X^, /d(^, 7'ij^) is convex in rj^. 

Suppose both rj and r'j G M;^ satisfy the constraints (I83> and ( I84> . and there exists some i* such that r[, ^ r". . 
We shall first show that r^. , r", are both finite. If not, without loss of generaUty suppose r^. = 00, which implies 
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that i?i. — oo. Now construct a new vector rj' such that r'l' = r- = oo if i = i* and r"' = ?■" otherwise. Note: 



we have r,-',' > r'l, . It is easy to check that r'^ satisfies the constraints (I83t and ( I84t (Note: we let oo — oo = 0). 

But we have 

1^ 1- cxp(-2rr) ^ 1 ^ 1 - cxp(-2rf ) _ 1 

which is contradictory to ( I92l i. 

Now let Ti = {r^ + r'/)/2 for all i G I^. Note that r^. is equal to neither r'^, nor r'/, since r^. ^ rf* and both 
are finite. It is obvious that rj^ G R;^. Furthermore, we have 

1 A 1 - exp(-rO 1 1 A 1 - cxp(-rO 1 A 1 - exp(-rf ) 

^ ^ ^^ - ^2^ ^^ 2^ ^2 ^^V 

"X i^i "Ni "X ^ j^i "Af, ^ ,;^i "N 

> T^TT^, (95) 



i?*(i?lj' 



and 



J2^^ - l^fDHRTji-^^r'j:J + -fD*(R:,j{A,rjJ 



(96) 



i£A 



> !d- {Rx^ ) {A rx^), V nonempty set A^Il- (97) 

Hence fi^^ satisfies the constraints ( I83> and ( I84> . Since l/cr^- + X]i=i(l ~ exp(— 2ri))/(T^. is a strictly concave 
function of rj^^, the inequality in ( I94> is strict, which results in a contradiction with ( I92> . 

Now only ( I55t remains to be proved. We shall first show that r*(i?ij^) == implies Ri = 0. Without loss of 
generality, suppose r\ {Rjj^ ) = 0. Then it is easy to check that ( I83> still holds if we set Rl = on its left hand side. 
So if Rl > 0, we can increase r^ {Rxi^ ) by a small amount without violating ( I83t and ( I84> . which is contradictory 
to the fact that r'^{Rxj^) is unique. Hence without loss of generality, we can assume r*{Rxi^) > for all i G II- 
Otherwise by restricting to the set G — {i ^ ^l ■ f*{RxL) > 0}' the following argument can still be applied. 

Since \5A\ holds, the righthand side of ( I83t becomes f{A, r'j {Rxl))- By ( I8Q> . it can be shown that if in ( I83t . 
the constraints on J2i£S ^i ^^'^ J2ieT^i ^"^ tight, then either 5 C T or T C 5. Let A = Ufcex At, where 
Ak{k G Xa') are the sets for which the constraints on J^ieA ^« ^^ '■^S^'- ^'^ * I83> . If there is no such an Ak, let 
A = 9. 

If ^ = Tl, we are done. Otherwise pick any i* G 2l\A. We can increase r*. (Rxl) to r*. (Rxl) + '^ for some 
(5 > without violating any constraints in ( I52> and ( I53> . which is contradictory to the uniqueness of r*, (Rxl)- 
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